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WWC Study Reports are intended to support decision making; neither the What Works Clearinghouse 
(WWC) nor the U.S. Department of Education endorses any interventions. No single Study Report should 
be used as a basis for making policy decisions because (1) few studies are designed and implemented 
flawlessly and (2) all studies are tested on a limited number of participants, using a limited number of 
outcomes, at a limited number of times, so generalizing from one study to any context is very difficult. 

To highlight these issues, the WWC Study Reports describe in detail the specifics of each study, focusing 
primarily on studies that provide the best evidence of effects (randomized controlled trials). Systematic 
reviews of the evidence will be conducted to summarize the results of the individual studies. 

See the WWC Brief Study Report (PDF) for a summary of this study. 



Topic: Curriculum-Based Interventions for Increasing K-12 Math Achievement — 
Middle School 



Intervention: Connected Mathematics Project 
Research Design: Quasi-Experimental Design with Matching 

Study Rating: o 



Date Released: September 15, 2004 

Summary of Results: Schneider found no statistically significant differences between the 

intervention and comparison groups on the school-level Texas 
Assessment of Academic Skills (TAAS) analyzed in the study. The findings 
may have been affected by only three schools implementing more than 
one-third of the total math problems in each year and grade. In addition, 
the sample size was small. 



= Meets Evidence Standards = Meets Evidence Standards with Reservations = Does Not Meet Evidence Standards 



The What Works Clearinghouse (www.whatworks.ed.gov) was established in 2002 by the U.S. 
Department of Education’s Institute of Education Sciences to provide educators, policymakers, 
researchers, and the public with a central and trusted source of scientific evidence of what works in 
education. Please email all questions and comments to info@whatworks.ed.gov . The What Works 
Clearinghouse is administered by the U.S. Department of Education through a contract to a joint 
venture of the American Institutes for Research and the Campbell Collaboration . 
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Intervention: Connected 
Mathematics Project 

Operational Features 

Between 1991 and 1997, the Connected 
Mathematics Project (CMP) received funding 
from the National Science Foundation to 
develop a mathematics curriculum for middle 
school (grades 6 through 8). The CMP, 
developed at Michigan State University, is 
reported to adhere to the National Council of 
Teachers of Mathematics (NCTM) Curriculum 
and Evaluation Standards. 

The key feature of the CMP is that it is a 
problem-centered curriculum that provides skills 
practice. The CMP is a comprehensive 
mathematics curriculum that has 24 units, with 
eight units at each of three grade levels. Each 
unit contains four to seven investigations. Each 
investigation in turn contains one to five 
problem sets and ends with a mathematical 
reflection activity. 

Support for implementing the CMP curriculum 
includes professional development prior to 
implementation and follow-up professional 
development each summer after 
implementation. In addition, pilot sites in this 
study provided regular meetings with their 
teachers throughout the school year. 

In this study, the intervention group consisted of 
23 schools that were using the CMP, and the 
comparison group consisted of 25 schools that 
were not using the CMP. For the comparison 
schools, Schneider does not provide any 
information on what curriculum they were 
using. 

People, Settings, and Timing 

This study was conducted in Texas. Although 
districts within Texas volunteered to pilot the 
CMP curriculum, implementation at the school 
and teacher level was not voluntary. 

Schneider’s study focuses on the schools in 
Texas that were using the CMP curriculum. 
Schools participating in the pilot CMP 
represented rural, suburban, and urban areas. 
Schneider reports that many of these schools 
have predominantly minority students. 



Three groups of students are the focus of this 
study. Group 1 students experienced the CMP 
curriculum from grades 6 through 8 between 

1996- 1997 and 1998-1999. Group 2 students 
experienced the curriculum in grades 6 and 7 in 

1997- 1998 and 1998-1999. Students in Group 3 
had the curriculum only while they were in 
grade 6 in 1998-1999. 

Cost Information 

Cost information on the CMP curriculum is 
provided on the publisher’s website 
www.phschool.com/math/cmp/index.html . The 
cost varies by the number of units purchased. 

Intended Duration 

The CMP is designed to be a year-long 
curriculum in grades 6 through 8. Schneider 
reports on the effects of varying length of 
curriculum implementation on three groups of 
students. Group 1 students had the curriculum 
for three years in grades 6 through 8. Group 2 
students had the curriculum for two years in 
grades 6 and 7. Group 3 students had the 
curriculum only in grade 6. 

Scientific Rationale 

Schneider does not provide a scientific rationale 
for this study. 

Overview of the Study 

Purpose 

The purpose of the study is to determine 
whether the use of the CMP, a new curriculum, 
affects student achievement in a state-mandated 
assessment. Because of high-stakes testing and 
accountability in Texas, districts prefer a 
curriculum that is aligned to the Texas state 
assessment test. Both the CMP curriculum and 
the state assessment are reported to be aligned 
with the NCTM standards. 

Intervention Fidelity 

Schneider surveyed the teachers in the target 
schools in this study to determine intervention 
fidelity. In most instances, only one teacher 
from a school responded for each grade level 
and year. Schneider reports that schools varied 
in their level of implementation. Although the 
CMP curriculum was taught throughout the 
whole year, no school reported having the goal 
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of completing all the problems at any grade 
level. 

Schneider also created a category of high-use 
schools — those that reported teaching more than 
one-third of the total number of problems for a 
grade and year. Schneider reports separate 
analyses for this group of schools. 

Outcome Measures 

The state assessment — Texas Assessment of 
Academic Skills (TAAS) — is the primary 
outcome measure used in the study. According 
to the Texas Education Agency, TAAS is a 
criterion-referenced assessment that measures 
problem-solving and critical-thinking skills and 
is aligned with the NCTM standards. Thus, for 
WWC purposes, this outcome measure has 
evidence of face validity and reliability and is 
properly aligned. 

This study reports two levels of data for the 
student achievement in the state assessment in 
the study: the school-level passing rates on 
TAAS and the student-level scores on the Texas 
Learning Index (TLI). 

Because the unit of intervention delivery was at 
the school level and the selection of comparable 
comparison schools was also done at the school 
level, this report focuses on the school-level 
TAAS passing rates and on the school-level TLI 
scores, which the author was able to provide in a 
follow-up. Unlike the student-level TLI statistic, 
the school-level TLI scores do not have the 
problem of dependence and clustering within 
schools and classes. 



Research Design 

The research design for this study is quasi- 
experimental with matching. The CPM pilot 
schools that sent 6th-, 7th-, and 8th-grade 
teachers to a six-day summer professional 
development program conducted by the Texas 
Statewide Systematic Initiative (SSI) in 1996, 
1997, and 1998 are the 23 intervention schools. 

To select the comparison schools, Schneider 
reports running a regression analysis on the 
demographic data, prior school-level TAAS 
passing rates, and school enrollment for both the 
intervention schools and the potential pool of 
comparison schools. Schools with the closest 
match on the predicted value on the current 
TAAS passing rate with the same grade levels 
encompassed by the intervention schools are 
those selected as comparisons. Based on these 
selection criteria, 23 matched schools were 
identified. Two more comparison schools were 
then added because they were feeder schools to 
two of the comparison schools. Thus, the final 
total number of comparison schools used in the 
study is 25. 

Schneider’s analysis is at the school level, 
which is the same level as the unit of 
intervention delivery. 

Participant Flow 

The study started with 23 intervention schools 
using the CMP curriculum and 25 comparison 
schools not using the CMP curriculum. The 
same number of intervention and comparison 
schools remained throughout the analysis. (See 
Ligure 1.) 
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Figure 1. Participant Flow (Schools) 
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Reference Periods 

The time frame for investigating the CMP 
curriculum in this study is school years 1996— 
1997 to 1998-1999. The teachers first received 
professional development in the CMP in the 
summer of 1996. Pretest measures used for 
matching are taken from years 1994, 1995, and 
1996. 

Baseline Data 

Schneider provides the following derived 
baseline data based on the pretest TAAS passing 
rates for each of the intervention and 
comparison schools. (See Table 1.) 



Table 1. Pretest Characteristics of the 
Study Sample 



Characteristic 


Intervention 


Comparison 


p Value 


TAAS passing 
rate: Mean (SD) 


71.7(16.97) 


72.4(16.68) 


N/A a 


Sample size 
(school) 


23 


23 b 


N/A a 



Note. TAAS = Texas Assessment of Academic Skills 

“Schneider did not test for pretest differences. 

b Schneider does not provide the TAAS passing rates for the two 

additional comparison schools that were added to the sample later in the 

matching process. 

Statistical Methods 

Schneider uses hierarchical linear modeling 
(HLM) to analyze data at the school level and 
across all schools. Schneider also reports 
separate analyses by group. In addition, 
Schneider provides a separate analysis of the 
three schools using the CMP curriculum that 
were classified as high implementation together 
with the corresponding comparison schools. 



Outcomes and Estimation 

Schneider uses HLM to determine significance 
of the school-level passing rate on the TAAS. 
Across all groups (with varying duration of 
intervention) and all schools, no statistically 
significant difference is found between schools 
using the CMP curriculum and the comparison 
schools on the TAAS passing rate for 
mathematics. When analysis is separated by 
group, no difference is found between schools 
using the CMP curriculum and comparison 
schools. The difference is also not significant 
between schools using the CMP curriculum 
designated as high-implementation schools and 
their matched comparison schools. 

In a follow-up, Schneider provided WWC with 
the posttest mean scores on the TLI for each 
intervention and comparison school and for each 
group. Table 2 shows the mean and standard 
deviations of the TLI posttest scores at the 
school level for each group. The effect sizes 
computed by WWC between the intervention 
and comparison groups are not significant. (See 
Table 3 and Figures 2a to 2c.) 

Cautions 

The sample size at the school level is too small 
to allow for sufficiently precise estimates of the 
effect size. 
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Table 2. Impact Reported by Schneider (2000): School-Level Posttest 
Scores on TLI Math Achievement 



Group (duration of 
intervention) 


Posttest scores on math achievement: TLI 
mean ( SD ) 


Intervention group 
estimated effect size 
(and significance) 


CMP Intervention 


Comparison 


CMP 


Group 1 (1997-1999) 


73.2 (6.37) 


72.9 (5.46) 


NR a 


Group 2 (1998-1999) 


72.3 (6.59) 


74.2 (5.29) 


NR a 


Group 3 (1999) 


73.6(5.41) 


74.5 (6.37) 


NR a 



Note. TLI = Texas Learning Index. 

a Schneider does not report effect sizes but reports that the difference was not significant in the hierarchical linear 
modeling analysis of school-level passing rates on the Texas Assessment of Academic Skills. 



Table 3. Impact Calculated by WWC: School-Level TLI Posttest 
Scores 



Group 
(duration of 
intervention) 


CMP 

Intervention 
group mean 
score (SD) 


Comparison 
group mean 
score (SD) 


Estimated Effect 3 


CI C 


p Value d 


Mean 

difference 


Effect size b 


Group 1 


73.2 (6.37) 


72.9 (5.46) 


0.30 


0.05 


±.61 


> .05 


Group 2 


72.3 (6.59) 


74.2 (5.29) 


-1.90 


-0.31 


± .62 


> .05 


Group 3 


73.6(5.41) 


74.5 (6.37) 


-0.90 


-0.15 


+ .66 


> .05 



Note. TLI = Texas Learning Index 

a Effect is shown both as the mean difference between the intervention and comparison groups and as a relative effect size, 
which is equal to the ratio of the mean difference to the standard deviation of the outcome measure. 
b All estimates of effect size are based on the d-index. 
c Confidence interval around the effect size estimate. 
d Reflects significance level of the effect size estimate. 
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Figure 2a. Impact Reported by Schneider (2000) : a School-Level TLI 
Posttest Scores in Group 1 
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Note. TLI = Texas Learning Index. 

a Confidence intervals are computed by the WWC. Schneider reports that the intervention group 
scores do not differ significantly from the comparison group scores on the Texas Assessment of 
Academic Skills. 
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Figure 2b. Impact Reported by Schneider (2000): a School-Level TLI 
Posttest Scores in Group 2 
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Note. TLI = Texas Learning Index. 

a Confidence intervals are computed by the WWC. Schneider reports that the intervention group 
scores do not differ significantly from the comparison group scores on the Texas Assessment of 
Academic Skills. 
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Figure 2c. Impact Reported by Schneider (2000) : a School-Level TLI 
Posttest Scores in Group 3 
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Note. TLI = Texas Learning Index. 

a Confidence intervals are computed by the WWC. Schneider reports that the intervention group 
scores do not differ significantly from the comparison group scores on the Texas Assessment of 
Academic Skills. 



Intervention Developer Contact 
Information 

Contact the local Prentice Hall Sales 
Representative at their general number 
1-800-848-9500; or contact CMP at 
517-432-2820 or visit the website 
www.math.msu.edu/cmp/index.html . 

Related Studies 

See reports on other studies of Middle School 
Math curricula. 

See reports on other studies of the Connected 
Mathematics Project . 



Report Production 

Date created: September 15, 2004 

Topic area reviewed under: Curriculum-Based 
Interventions for Increasing K-12 Math 
Achievement — Middle School 
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WWC Study Ratings 3 : Schneider (2000) 

Causal Validity: Meets WWC Standards with Reservations, a Quasi-Experimental Design with 

Matching and No Attrition or Disruption Problems 

Intervention schools were matched on relevant variables to similar comparison schools in Texas. Pretest differences between 
the two groups after the selection of the matched comparison schools were not addressed by the author. There was no attrition 



at the school level, and no extraneous events were identified that appeared to confound the intervention’s effect. 


Other Study 
Characteristics 


Study Rating 


Study-Specific Information 


Intervention Fidelity 


• • 


The CMP intervention is well implemented and designed and meets the definition of 
middle school math. Schneider also provides enough information about the curriculum, 
the population, and the setting to allow replication. Schneider states that no schools 
intended to implement all the units of the curriculum in a single grade and year, yet 
Schneider does not report any deviation of the intervention schools from the way the 
curriculum should be implemented. 


Outcome Measures 


• • 


Although Schneider reports both school-level and student-level achievement outcome 
measures of the Texas Assessment of Academic Skills (TAAS), only the school-level 
outcome measure is reviewed in this report because the CMP curriculum was 
implemented at the school level. The primary outcome measure shows evidence of face 
validity and reliability and is properly aligned. 


People, Settings, and 
Timing 


• 


Although Schneider reports that the schools in the sample represent rural, suburban, 
and urban settings, other variations in people and settings that WWC deems important, 
such as sex, tracking, and class size, are not reported. The outcome measure was given 
some time in April, just before the end of the school year. 


Testing within 
Subgroups 


• 


The intervention’s effect was tested across the entire sample but not within important 
subgroups. 


Analysis 


• 


Schneider’s analysis is at the school level, which is the same level as the unit of 
intervention delivery. The sample sizes are too small at the school level to allow for 
sufficiently precise estimates of the effect size. 


Statistical Reporting 


• • 


The statistical tests are adequately reported. Sufficient details such as means and 
standard deviations are provided so that effect sizes can be estimated for the outcome 
measure of interest. 



Note. •• Fully meets criteria; • Meets minimum criteria; X Does not meet criteria. 

a For more information on the criteria used to rate this study, see the WWC Evidence Standards: Middle School Math . 
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